Methods, for testing a control apparatus with a test device via a signal interface have been carried out in industrial practice since control apparatuses have been used and tested for their functionality.
A control apparatus denotes any computer-based apparatus or other technological equipment programmable with some functionality. Such a device could be embodied in hardware that is programmed once to perform a discrete function or a more general purpose device that can be re-programmed multiple times. The control apparatus is usually designed to be connected to a physical process where it acts on the process and/or monitors the process by receiving measurement data via a signal interface. The control apparatus can be embodied, for example, in a single board computer with appropriate I/O functionality, or it can be a more complex distributed computer system, or it can be embodied by other arrangements known or later developed in the art. The systems and methods disclosed herein are applicable to control apparatuses without regard to their actual hardware equipment and apparatus-technological design.
The number of functionality of control apparatuses, generally, can be immense—and are continuously increasing. For example, control apparatuses used in motor vehicles can easily comprise several hundred or even several thousand individual functionalities.
The performance of a functionality that is expected from the control apparatus—i.e., the target performance functionality—is often obtained, for example, from functional specifications and other specifications used in a particular application. A challenge arises in ensuring that the actual implemented operational performance of the control apparatus functionality corresponds with the desired and predetermined target performance functionality. In order to obtain the intended performance, target performance functionalities are analyzed, and translated into a signal level requirement. The signal level requirements represent the signals required for the control apparatus to be moved into a state that allows the generation of an expected reaction when certain conditions are present. Specifically, the reaction should be the one predetermined by the target functionality of the control apparatus, compliance with which can usually be observed according to the observed signal levels. Thus, the test of the control apparatus consists of the translation of the test cases into input signal patterns, the application of signal patterns by the test device to the control apparatus via the signal interface, and the acquisition of the pertinent state variables of the control apparatus by the test device. Then the comparison of the actual performance functionality and the target performance functionality naturally leads to the conclusion of interest indicating whether the control apparatus has the desired target functionality.
The analysis of the target functionality, the manual conversion of the target functionality into requirements, the additional manual conversion of the requirements in test cases—unequivocal test situations into which the control apparatus is put—and the translation of the test cases into signal patterns are extremely time consuming procedures that are susceptible to error. Thus, there is a general desire to automate these steps as much as possible when testing a control apparatus.
Different attempts are known in the art to automate methods for testing a technical device—but not actually a control apparatus—in particular, to acquire the target functionality and translate those targets into formal requirements and test cases, from which the actual test process then results.
A comparatively complicated method is known from the article “Obtaining Models for Test Generation from Natural-language-like Functional Specifications” by M. W. Esser and P. Struss (in G. Biswas et al. (eds), DX'07, 18th International Workshop on Principles of Diagnoses, May 29-31, 2007, Nashville, USA). This method relates to test case generation for testing software in the automotive sector. Here, requirements are obtained in formal language by a selection of gap texts that are suitable to take into account the typical requirement, usually using conditional constructions (e.g., if-then) and time dependencies through the acquisition of the duration of the states. The filled-out gap texts provide a formal representation of the target performance functionality, which describes the system (e.g., control apparatus) in the error-free case. Subsequently, error models are derived from the error-free model (e.g., ok model). This is done by observing the signal patterns that result from the intentional introduction of the error and comparing the behavior of the ok model from that of the error model. This, however, is comparatively complicated, because in addition to the ok model, an error model must also be generated for each error.
Another method for testing a technical system is disclosed in the article “AI-Planner Assisted Test Generation” by A. K. Amschler Andrews et al. (in Software Quality Journal, 10, 225-259, 2002, Kluwer Academic Publishers). In this article, the automatic test case generation for a data tape silo device is described, where the target functionality of the apparatus is acquired based on a model—namely by a state automaton—but time dependencies are not taken into account.
However, accounting for time dependencies can be essential for testing numerous control apparatuses because the implemented functionality often presents time dependencies, and the test of a control apparatus is incomplete, and therefore of low validity, if the time dependencies are not taken into account.
The present invention discloses exemplary embodiments that provide a method for testing a control apparatus that improves the acquisition of the target performance functionality of a control apparatus, while taking into account the time dependency of the target functionality during the test case generation.
One embodiment of the present invention discloses a method for testing a control apparatus with a test device, by carrying out the following process steps:
acquisition in formal language of at least one time dependency-containing target functionality as requirement, where the target functionality comprises at least one action and at least one expected reaction,
generation of a requirement model from the requirements,
generation of test cases by at least one achievability analysis carried out in the requirement model taking into account the time dependency of the target functionality, and
translation of the test cases into signal patterns, application of the signal patterns by the test installation to the control apparatus via the signal interface, and acquisition of the pertinent state variables of the control apparatus with the test device.
One potential advantage to the disclosed methods for testing a control apparatus is that they allow the direct testing of the target functionality of a control apparatus, taking into consideration time dependencies of the target functionality, without an error model of the target functionality.
The use of formal language to acquire the time dependency-containing target functionality is semantically unequivocal, thus avoiding the lack of precision associated with common speech. This, however, does not mean that the formal language is not comprehensible in common language.
A target functionality typically comprises at least an action and at least one expected reaction. One example of an action can be the pressing of a button. The reaction of the target function that the action causes can be fixed. For example, the action of pressing a button might be that an alarm is activated at the latest after 100 ms. In this example, the “at the latest after 100 ms” is the required time dependency of the target functionality. While the functional scope of the control apparatus can naturally also include target functionalities that present no time dependency, the present invention can be employed when at least one target functionality of the control apparatus comprises a time dependency.
A requirement model that represents the totality of the requirements serves as the foundation for an achievability analysis. The problem of the achievability analysis is to find a sequence of actions through which the tested control apparatus can achieve a certain end state that is possible only if the control apparatus has the required target functionality. The achievability analysis can make use of all the actions that comprise the requirement model. It is particularly important here to take into account the existing time dependencies of the target functionality. Accordingly, a causal sequence of actions and reactions maintained and admissible time intervals for the actions and reactions are included in the achievability analysis.
According to a preferred embodiment of the method according to the invention, the time dependency of the target functionality consists of presetting a reaction time interval of the expected reaction, which accommodates the actual behavior of many technical systems. Thus, for example, a target functionality can be defined such that the reaction to an action is admissible and error free only if the reaction occurs only in a certain time interval after the triggering action.
For example, assume the target functionality of switching on a car alarm after the vehicle is locked (action), at least after a delay time tx, but at no later than a delay time of ty after the triggering event. These conditions are close to actual practice because the response times of a control apparatus to a certain action can depend, for example, on the current state of the control apparatus, its failure, the priority of the action, or the priority of actions that occurred earlier in time. The response time can also depend on temporally nondeterministic components of the control apparatus, such as, for example, nondeterministic field buses.
In a most particularly preferred embodiment of the invention, the achievability analysis occurs with the allocation of at least one time boundary condition. Further, an additional time boundary condition ensures that the complete reaction time interval of the target functionality is taken into account. This occurs preferably by the presetting of the limit of a time inequality or by the presetting of this time inequality. This measure has the highly advantageous effect that test cases can be generated by the achievability analysis that take the reaction times into account, completely and without error.
The above-mentioned example of the locking of a vehicle (action) can illustrate this usage. In this example locking time can be called tstart. The reaction time interval is [tstart+tx; tstart+ty] according to the above example. If, in the context of the achievability analysis, an additional time dependency is taken into account, which concerns the reaction time of this first event—activation of the alarm—conflicts can occur easily. Such a time dependency could consist, for example, of the requirement that the time of the activation of the car alarm—hereafter called te—follows after a defined, but not yet temporally fixed, time tz. When this time tz, which is defined, but temporally not yet concretely fixed, is in the reaction time interval [tstart+tx; tstart+ty], this leads to a portion of the admissible reaction time interval [tstart+tx; tstart+ty] no longer being taken into account. Therefore, one must ensure that this additional boundary condition te>tz is completed by an additional chronological order condition, for example, by the requirement that tz is elapsed before the start of the reaction time interval [tstart+tx; tstart+ty], i.e., the time inequality tz<tstart+tx holds.
It is advantageous to proceed, if needed, in such a way that the reaction time te precedes a certain, but actually not yet fixed, time tz, where here, to prevent time conflicts, the additional time boundary condition must be provided that the time tz must occur after the reaction time interval; consequently tz>tstart ty holds.
The above-explained method for introducing additional time boundary conditions to take into account the complete reaction time interval is particularly advantageously usable if the method according to the invention generates a test case by achievability analysis in the formal requirement model, by
first establishing, in a start plan, an initial state or an initial action (start), and an end state or an end action (end), in each case by the selection of at least one state variable with defined state,
inserting, in the start plan, a requirement from the requirement model, between the initial state or the initial action (start), and the end state or the end action (end), so that a partial plan is created, and where the resulting causal coverage is acquired, particularly by causal connections, and
by optionally inserting in the partial plan successive additional requirements from the requirement model, until, a solution plan, i.e., a causally gap-free path is achieved between the initial state (start) and the desired end state (end).
The acquisition of the requirement in formal language, which occurs at the beginning of the method, is preferably carried out by means of a controlled natural language.
This has the advantage that, in spite of the use of lexical, grammatical and semantic restrictions, the formulation of the requirements remains comprehensible in common language.
The problem is solved furthermore with a computer program with programming code means, for the purpose of carrying out the steps of the above-mentioned method. The program can be implemented in a data processing system, such a computer program with programming code means that are stored in a computer readable data carrier.
According to an additional teaching of the invention, the above-presented problem is also solved with a test device for testing a control apparatus by designing the test device in such a way that it allows the above-explained method to be carried out on the control apparatus.
In detail, numerous possibilities now exist to embody and further vary the method according to the invention and the test device according to the invention.